Papers by Benjamin K. Bergen
Word Acquisition in Neural Language Models (2022.tacl-1)
Copied to clipboard
| Challenge: | Language models acquire individual words during training, based on unigram token frequencies, before transitioning loosely to bigram probabilities, eventually converging on more nuanced predictions. |
| Approach: | They examine how neural language models acquire individual words during training, extracting learning curves and ages of acquisition for over 600 words on the MacArthur-Bates Communicative Development Inventory. |
| Outcome: | The models follow consistent patterns during training for both unidirectional and bidirectional models, and for both LSTM and Transformer architectures. |
Do Language Models Make Human-like Predictions about the Coreferents of Italian Anaphoric Zero Pronouns? (2022.coling-1)
Copied to clipboard
| Challenge: | Some languages allow arguments to be omitted in certain contexts, but human language comprehenders construct expectations about which referents are more likely. |
| Approach: | They ask whether Neural Language Models extract expectations from sentences with zero pronouns from five behavioral experiments conducted in italian by Carminati (2005). |
| Outcome: | The results suggest that human expectations about coreference can be derived from exposure to language, and also indicates features of language models that allow them to better reflect human behavior. |